Subject: Problems we are solving

From: [email-masked]
Date: Thu, 22 Jul 93 17:51:25 EDT

Since recent discussions have focused on certain problem scenarios, I
thought I might throw out a "reference configuration" and try to create
a list of different problems people have mentioned or which I've thought
about.  I'm using the terminology I suggested Tuesday.

Key:
Rx = Router x, Sx = (End) Station x (mac+sap), Px = Port x
TC = transport connection, identified by router pair Rx-Ry
cr = circuit, identified by station pair Sx-Sy

       +-------+ TC R1-R2                    +-------+
S1--L  |  R1  Pc-----------------------------Pa R2   |  L--S5
    A--Pa      |  carries cr's S1-S5, S1-S6  |      Pb--A
S2--N  |       |               S3-S5, S4-S6  |       |  N-S6
       |       |                             +-------+
       |       |
S3--L  |       |  TC R1-R3                   +-------+
    A--Pb     Pd-----------------------------Pa R3   |  L--S7
S4--N  |       |  carries cr's S1-S7, S2-S8  |      Pb--A
       +-------+               S3-S7, S4-S7  |       |  N--S8
                                             +-------+

Here are some problem scenarios - specific examples of all the general
cases I could think of.
1 - S1 RNRs circuit S1-S5.  If this is not reflected to S5 in a timely
    way, the following could happen (depending on R1's buffering and
    queueing implementation):
      a - the other circuits sharing TC R1-R2 start to slow down in
          the right-to-left direction (e.g., because TCP flow control
          starts knocking down the R1 receive TCP window)
      b - the circuits on TC R1-R3 could also start to slow if R1
          resources they need are getting eaten up
    Without circuit-level messages, R2 can only see TCP backing up and
    doesn't know to RNR S1-S5 versus S3-S6.  Plus when TCP is backed
    up, the other circuits have already suffered.

2 - The LAN attached to R1-Pb is temporarily disabled (e.g.,token-ring
    beaconing).  Depending on R1's implementation, this could cause
    slow down of both TCs' traffic into R1, or just the circuits to
    S3 and S4.

3 - Transport connection R1-R3 gets very slow because of internal IP
    network congestion (let's say both directions).  Both R1 and R3
    will get congested unless they exert back pressure on the links
    feeding that TC.  R1's design may cause traffic on TC R1-R2 going
    into R1 to also slow down if this is not done quickly enough.

4 - S5 and S6 overrun TC R1-R2's link capacity, R2 gets congested.
    This is another case where end stations may need to be RNRd,
    effectively the same as problem #3.

5 - Buffer resources in R3 are shared with other protocols and are not
    available to DLSw when a burst of traffic comes from S7 or S8.
    Another case where end stations may need to be RNRd.

I support the following general principles.  Some implementations
may not choose to achieve these, but I think the architecture we are
defining needs to give implementations the tools to achieve them.

1 - congestion on one circuit's path should not affect (really, have
    the minimum possible effect on) other circuits on the same TC, or
    circuits on other TCs
2 - congestion on one TC should not cause other TCs in the same
    router to become congested
3 - port-level conditions should be reflected only on circuits
    using that port
4 - flow control should be exercised independently in each
    direction, whether one is reflecting circuit or TC-level
    conditions

One more principle I'd like to promote is that a data link belongs to
the end station and its adjacent router, not to the router on the
other side of the WAN.  So I think one router can tell another "stop
sending to me on this circuit", but not "go send RNR to the end station
you are attached to on this circuit".  I also think that the problems
numbered 3-5 are outside the scope of what our flow control architec-
ture should address.

I mention these principles because I think they lay the following
requirements on our architected flows:

- We must have circuit level flow control, otherwise we have no circuit
  fairness mechanism and no port-level mechanism.  So I disagree that
  watching TCP queues is sufficient.  Robin Layland said, referring to
  circuit-level flow control, "it may be in the RFC..." - Robin, it's
  not in the RFC, but one flavor of it was in David Miller's append
  which Shannon Nix sent out at the start of this mailing list.
- We must have unsolicited control flows (need quick reaction to come
  close to achieving fairness).  Whether we call these ENTERBUSY or
  whatever, it is up to the receiving router exactly when to do the RNR.
- If we have a TC-level control flow (e.g., "don't send me any more
  traffic on any circuit in this TC"), it's up to the receiving router
  whether to RNR all data links then or only those that try to send in.

Robin, I did not understand your last point about sending RNR with
a certain sequence number.  Could you please restate the scenario with
a few more specifics?  I got confused about which router was sending
what to whom.  You could use the configuration pictured above if you
want.

Peter Gayek